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(57) Abstract 

The present invention is related to an image sensor 
comprising an array of rows (i) and columns (j) of pixels 
(Xjj), all the pixels of one column of the array being 
connected to at least one common pixel output line (lj) 
having at least one memory element (Mj) and at least a 
first amplifying element (Aj), all these amplifying elements 
(Aj) being connected to a common output amplifier (D). 
According to one preferred embodiment, said image sensor 
comprises: a second amplifying element (Bj) on the output 
of the memory element (Mj); said common output amplifier 
(D) having at least two input terminals; means (SI) for 
switching the pixel's signal on the common output line (lj) 
and the memory element's signal (Mj) to respectively third 
and second amplifying element (Aj and Bj) of one column; 
and means (S2) for switching the two output signals of the 
amplifying elements (Aj, Bj) of one column to respectively 
first and second input terminals of said common output 
amplifier (D). 
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DEVICES AND MET HODS FOR IMPROVING THE IMAGE QUALITY IN AW 

IMAGE SENSOR 



10 



Field of the invention 

The present invention relates to solid state 
imaging devices being manufactured in a CMOS- or MOS- 
technology. 

15 Mo r e particularly, the principal object of 

the present invention is related to methods and devices 
which are able to improve the image quality in an image 
sensor. 

Another object of the present invention is 
20 related to the improvement of the image quality by a method 
of correcting isolated pixel values present in an image 
taken by imaging devices . 

Backgroun d of the invention 
25 Solid state image sensors are well known. 

Virtually all solid-state imaging sensors have as key 

element a photosensitive element being a photoreceptor, a 

photo diode, a photo transistor, a CCD gate, or alike. 

Typically, the signal of such a photosensitive element is a 
3 0 current which is proportional to the amount of 

electromagnetic radiation (light) falling onto the 

photosensitive element. 

A structure with a photosensitive element 

included in a circuit having accompanying electronics is 
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called a pixel 1 Such pixel can be arranged in an array of 
pixels so as to build focal plane arrays of rows and 
columns . 

Commonly such solid state image sensors are 
5 implemented in a CCD- technology or in a CMOS- or MOS- 
technology. Solid state image sensors find a widespread use 
in devices such as camera systems. In this application a 
matrix of pixels comprising light sensitive elements 
constitutes an image sensor, which is mounted in the camera 

10 system. The signal of said matrix is measured and 
multiplexed to a so-called video-signal. 

Of the image sensors implemented in a CMOS- 
or MOS- technology, CMOS or MOS image sensors with passive 
pixels and CMOS or MOS image sensors with active pixels are 

15 distinguished. An active pixel is configured with means 
integrated in the pixel to amplify the charge that is 
collected on the light sensitive element. Passive pixels do 
not have said means and require a charge sensitive 
amplifier that is not integrated in the pixel. For this 

20 reason, active pixel image sensors are potentially less 
sensitive to noise fluctuations than passive pixels. Due to 
the additional electronics in the active pixel, an active 
pixel image sensor may be equipped to execute more 
sophisticated functions, which can be advantageous for the. 

25 performance of the camera system. Said functions can 
include filtering, operation at higher speed or operation 
in more extreme illuminations conditions. 

Examples of such imaging sensors are 
disclosed in EP-A-0739039, in EP-A-0632930 and in 

30 US-A-5608204 . The imaging devices based on the pixel 
structures as disclosed in these patent applications 
however are still subject to deficiencies in the image 
quality of the devices. 



BNSDOCID- <WO 991623SA1_L> 



WO 99/16238 PCT/BE98/00139 



A problem in these CMOS based imaging devices 
appears because material imperfections and technology 
variations have as effect that there is a non-uniformity in 
the response of the pixels in the array. This effect is 
5 caused by a non- uniformity or fixed pattern noise (FPN) or 
by a photoresponse non- uniformity (PRNU) . Correction of the 
non-uniformity needs some type of calibration, e.g. by 
multiplying or adding/subtracting the pixel's signals with 
a correction amount that is pixel -dependent . 
10 Several methods to cancel FPN are based on 

techniques that are often called correlated double sampling 
or offset compensation. The methods in general are based on 
the following: the signal of the pixel is subtracted from 
the signal of the same pixels in a reference state (this 
15 reference state is typically the reset or dark state) . The 
difference of both signal is free of pixel-related non- 
uniformity. However, if the differencing circuit is common 
for a part of the imager (typically, common for one 
column) , a new non-uniformity will originate due to the 
20 non -uniformity of the differencing circuits. In a typical 
APS imager with common column buffers or column amplifiers, 
the new fixed pattern noise is column dependent, and is 
visible in the image as a shade of vertical stripes. 

A stripe- shaped FPN is much more annoying. 
25 than a pure statistical FPN. It is seen in experiments that 
a true random FPN of 5% RMS is barely visible to the human 
eye, whereas a stripe -shaped FPN remains visible even when 
the amplitude is below 1% RMS . The reason is that the human 
eye has a kind of built-in spatial filter that recognises 
30 larger structures even when they have low contrast. 

Even if in the case that we have no fixed 
pattern noise, the photoresponse non-unif ormity can be 
different from 0 . 
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Another problem arises due to processing 
imperfections, statistics, etc. This means that typically, 
a finite number of pixels in a pixel array will be 
defective (hard faults) or yield a signal that deviates 
5 visibly from the "exact" pixel value. Such faults appear as 
white or black (or grey) points in the image. This class of 
faults in the sequel is called an isolated pixel value. 

A known way to cancel these spots is to store 
a list of them and of their positions in the image in a 

10 memory unit in the device. In an image processing step, the 
isolated pixel value is then replaced by e.g. the average 
of the surrounding pixels. This method is viable, but has 
the disadvantage that it requires a memory. Moreover, it 
cannot handle isolated pixel values that appear 

15 intermittently or only in certain cases. A good example, is 
a so-called dark current pixel. Such pixels will appear 
white in a dark environment, but will behave normal in a 
bright environment . 

Other ways to cancel isolated pixels faults 

20 have been proposed, e.g. the spatial median filter or other 
types of Kalman filters can be used to remove such isolated 
faults. Unfortunately, such filters do also remove useful 
detail from the image. Consider the image of a star covered 
sky with an image sensor that has some faulty pixels that 

25 appear white. The quoted filters are not able to remove the 
white point due to faults, and leave the white points that 
are stars untouched. 



Aim of the invention 

30 The present invention aims to suggest a pixel 

structure and methods to improve the image quality, more in 
particular the image non- uniformity of in array of pixels 
by cancellation of the appearance of column-shaped fixed 
pattern noise (FPN) . 
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Main characteristics of the present invention 

As a first object, the present invention is 
related to an image sensor comprising an array of rows and 
columns of pixels, all the pixels of one column of the 
5 array being connected to at least one common pixel output 
line having at least one memory element and at least a 
first amplifying element, all these amplifying elements 
being connected to a common output amplifier . 

According to one preferred embodiment, the 
10 image sensor further comprises : 

- a second amplifying element on the output of the memory 
element, 

- said common output amplifier having at least a first and 
a second input terminals, 

15 - means for switching the pixel ■ s signal on the common 
output line and the memory element's signal to 
respectively first and second amplifying elements of one 
column, and 

- means for switching the two output signals of the 
20 amplifying elements of one column to respectively first 

and second input terminals of said common output 
amplifier. 

Preferably, the switching means comprise at 
least one cross-bar switch. 

25 According to another preferred embodiment, 

the image sensor further comprises before the amplifying 
element two parallel circuits being connected through 
switches to the common pixel output line, at least one 
circuit having said memory element. Preferably, both 

30 circuit have a memory element. 

According to another preferred embodiment, 
said common pixel output line is being connected through 
switches to said memory element and said amplifying 
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element, where the offset of the amplifying element is 
stored on the memory element during a first phase of the 
read-out, and this offset is subtracted from the signal of 
the amplifying element during the second phase of the read- 
5 out . 

The present invention is also related to a 
method of reading out an array of rows and columns of 
pixels in an image sensor as described hereabove according 
to the first embodiment, comprising the steps of: 
10 - amplifying the output signals of essentially each pixel 
of one column in the first amplifying element thereby 
obtaining a set of amplified pixel output signals, 

- amplifying the reference signals of essentially each 
pixel of one column in the second amplifying element, 

15 thereby obtaining a set of amplified pixel reference 

signals, 

- consecutively, for essentially each pixel of said column 
imposing the amplified pixel output signal to a first or 
a second terminal of said common output amplifier and 

20 imposing the amplified pixel reference signal to a 

second or a first terminal of said common output 
amplifier, while switching the amplified pixel output 
signal to respectively said first and said second 
terminals for essentially each consecutive pixel of said' 
25 column, said reference signal being imposed to the other 

terminal of said common output amplifier. 

A voltage can be imposed to a node. In this 
case, it means e.g. that the node is connected to a voltage 
source- The voltage source should be higher than the node, 
30 i.e. it should have a lower impedance. 

According to another preferred embodiment, 
the present invention is related to a method of reading out 
an array of rows and columns of pixels in an image sensor 
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as described hereabove in the second embodiment of the 
present invention, comprising the steps of : 

- sampling the signal in a first phase and storing it in a 
memory element, 

- sampling the signal in a second phase and possibly 
storing it in another memory element, 

- subtracting said first signal from said second signal in 
a unique output circuit. 

According to another preferred embodiment, 
the present invention is related to a method of reading out 
an array of rows and columns of pixels in an image sensor 
as described in the third embodiment of the present 
invention, comprising the steps of : 

- during a first phase, calibrating the output of the 
amplifying element to a predetermined value, 

- storing said value in a memory element during the 
application of a first signal of said pixel on the line, 

- during a second phase, applying a second signal of said 
pixel on the line in order to have on the output a signal 
proportional to the difference between first and second 
signals . 

Another aspect of the present invention is 
related to a method of replacing an isolated pixel value in 
the image of an image sensor, being an array of pixels, and 
wherein at least one current source is connected to the 
pixels, the method comprising the step of : 

- limiting said isolated pixel value between or to an 
upper and/or a lower bound that is derived from the 
values of pixels in the immediate neighbourhood of the 
said isolated pixel value. 

Preferably, said upper and/or lower bounds 
are found by extrapolating the immediate neighbourhood 
pixel values towards a value that corresponds to the 
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position of said individual pixel in relation to the 
immediate neighbourhood pixels. 

Said upper and/or lower bounds are found by 
extrapolating the values of a neighbour (V;iN) of the pixel 
having said isolated pixel value and of the neighbour 
thereafter (V 2 N) , the replacing pixel value being 
calculated as V x N+n* (V^-Z^N) , n being a real number. 

The neighbourhood and the neighbourhood 
thereafter are on the same row of said array. Preferably, 
the upper bound is the maximum of a set of values, said set 
being determined as the pixel values (a,b,c,d,e) of pixels 
on the same row of said array as said isolated pixel , said 
upper bound being calculated as 

cmax = F(a,b,c,d) , or cmax G(E(a,b), E(e,d), E(b), E(d)) 
where F is a non-linear or linear function, G is a non- 
linear GE or linear function, E is an extrapolating 
function, wherein cmax= MAX (2b-a, 2d-e, b, d) 
together with 

cmin= MIN(2b-a,2d-e,b,d) 
with MAX () yielding the maximum, respectively the minimum 
of the arguments, the corrected c-value being obtained as 

c=MIN (MAX (c, cmin) , cmax) . 

Brief description of the drawings 

Figure la represents a particular implementation of a 

column FPN cancellation method and the 
corresponding image sensor structure 
therefor. 

Figure lb represents an embodiment of a cross-bar 

switch used in the structure represented in 
Fig. la. 
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Figure 3 



Figure 4 



Figures 5 



Figure 2 describes another particular implementation 

of a column FPN cancellation method and the 
corresponding image sensor structure 
therefor. 

describes a particular embodiment of an 
output block being used in the structure as 
represented in Fig. 2. 

represents the switching diagram for the 
read-out signal applied to the several 
switches in the structure as represented in 
Fig. 2. 

represent another particular implementation 
of a column FPN cancellation method and the 
corresponding image sensor structure 
therefor, wherein Fig. 5a represents more 
particularly one column in an array of pixels 
being connected to two different structures 
represented in details in Fig. 5b and 5c 
during a first phase and a second phase, 
represents the specific topology used for the 
structure represented in Fig. 5b and 5c. 
represents the switching diagram for the 
read-out sequence to the several switches 
used in the structure represented in Fig. 6. 
Figures 8a and 8b represent a method of correcting 
isolated white pixel values being present in 
an image composed by an array of pixels. 



Figure 6 
Figure 7 



Brief descri ption of p referred embodiments of th«* p resent 
30 invention 

As a first object of the present invention, a 
first structure of an APS image sensor is represented in 
Fig. l. References (X x j ) , (x 2j ) and (X 3j ) are three pixels 
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of a column of an image sensor. The pixel's signal on a 
common output line (lj) is represented in particular by the 
column bus "K" in the present case and is fed to the 
optional buffer amplifier Aj , and/or . stored on a memory 
5 element (capacitor C + switch S3), and fed to amplifier Bj . 
By the relative timing of the addressed pixel's reset pulse 
and the control of the switch S3, one can make that the 
pixel's signal and its reference level are available on 
amplifiers A j , reps. Bj . The fact that the signal passes 

10 through the column amplifiers Aj and B j , is a source of 
offset non-uniformity, which is column related and causes a 
vertical stripe-shaped FPN. More specifically, each column 
will feature an average "OV" offset voltage referred to the 
average of the other columns . 

15 Switches <S1) and (S2) are crossbar switches. 

Suppose that they are in the forward direction either in 
crossed directions. Both switches SI and S2 operate 
synchronously. In both cases, the signal on the capacitor C 
goes to the input of the output amplifier, and the signal 

20 on K goes to the + input of the output amplifier. Yet, the 
"OV" of the column will be positive in the one case and 
negative in the other case. Since the switches (SI) / (S2) 
are modulated, e.g. essentially turned direction at each 
new row (i = l, 2, 3) of the image, the average offset of a 

25 column will be zero. For each individual pixel of a column, 
there will be indeed remain an offset which is + or - OV 
but this is a very small feature, and is not recognised by 
the eye as a stripe. 

Another embodiment of the present invention 

30 is to suggest a read-out scheme for image sensors that 
suppresses the effects of non-uniformities caused by 
variations in pixels as well as variations in the output 
amplifiers. This read-out scheme can be used in sensors 
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that provide the output signal in terms of a difference. 
For example, in a. sensor with integrating pixels, this 
difference is the voltage between the output when the pixel 
is on reset state and the output voltage after integration 
5 time. 

Namely the method suggests to subtract the 
signal when a pixel is reset from the signal after the 
integration time, in order to have a signal which is free 
from pixel variations. In order to avoid the introduction 

10 of the non-uniform column amplifiers effects, the signal of 
the reset state as well as the signal after integration are 
sampled and held by the column read-out circuit. Finally, 
the subtraction is being carried out by a unique subcircuit 
at the sensor's output (D) . This is detailed in Fig. 2. 

15 For every row of pixels the read-out process 

is, of course, identical. Let us assume that the i th row is 
selected. When pixels are reset, the switches controlled by 
the signal Ol, are closed, thus the reset -level output of 

every pixel on the i th row, namely x , is stored on the 
2 0 corresponding memory element M r j (which is in the present 
case a capacitor) . 

Then, the switches S4j controlled by Ol are 
opened and pixels start integrating the charge carriers 
produced by the impinging light. After the lapse of the 
2 5 integration time, the switches S5j controlled by 02 are 
closed, thus storing the values of the pixel output to the 
memory element M s j (also a capacitor) . This value, for the 

pixel with coordinates i j is denoted as x . 

After the sample and hold phase for the two 

30 voltages, X(f and x& , for the first column and by 

proceeding from the first column to the last, the 
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appropriate column read-out circuit is connected with the 
output . 

When the i th column has been selected, the 
signal <I>3 drives the switch S6j to led the signal x r to 

- 5 the output modified according to the action of the column 
amplifier, so as a signal 

y^AjXy + Bj (1) 

is led to block D (common output amplifier) . 

The block D is an easily realisable circuit 
10 with an output 

z (n) = ty (n) - y (n-1) ] (2) 
where y(n) denotes the input as instant n. 

A preferred embodiment of such output block 
(D) is described in details in Fig. 3. Therefore, the 
15 output signal will be free from variations in the 
characteristics of pixels and the column amplifiers. 

y r ij~y s ij -Aj(x^xi) o) 

where Aj is easily reproducible, by example by using source 

followers as the local final phase of the column circuits 
20 (when Aj = 2 , B j = -V th j ) . 

Fig. 4 shows a switching diagram for the 
above-mentioned read-out circuit wherein 0>3 is controlling 
S4 j , 0>4 is controlling the switch S5j and 05 is 
controlling the switch S6j . 

25 According to another embodiment of the 

present invention, an attempt to overcome the problem of 
offset introduced by the column in a image sensor 
consisting of pixels is described in Figs. 5 & 6, which can 
be used with pixels that are read-out twice in every 

3 0 access. For example, in integrating pixels, one read 
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operation is being performed when the pixel is set on the 
reset phase (first phase) and the second read-out moment is 
after a certain integration time (second phase) , the first 
phase is defined by a period wherein the read-out signal of 
5 the pixel is according a first state while the second phase 
is defined by a period such as an integration period where 
the read-out signal of the pixel is in another state. 

In the first phase, when pixel output is V lf 
the capacitor C stores a charge : 
10 Q = C(V ac -V out ) = C(V ac -V 1+ V th ) (4 ) 
where V^h is the threshold voltage of \i. 

During second phase, when pixel output is V 2 , 
the capacitor stores again charge Q which now can be 
expressed as : 

15 Q = C(V gM -V 2 ) (5) 
where Vg^ is the gate voltage of /i . 

From (4) and (5) , we obtain : 
v g/z = V 2-V 1+ V ac +V th (6) 
Therefore : 

20 V out = V 2 -V 1+ V ac (7) 

i.e. the output voltage does not depend on the V t h (where 
variations in the V th cause offsets in the signal V out ) . 

The column amplifier can be implemented by 
using the topology described in details in Fig. 6. 

25 Switches S7, S8, S9, S10 and Sll are being 

controlled by signals OS and 06 (when <Di is high, the 
corresponding switches are closed) as represented in 
Fig. 7. 

According to a preferred embodiment 
3 0 represented in Fig. 7, S7, S8 and S9 are controlled by d>5 
while S10 and Sll are controlled by 0>6 (when Oi is high, 
the corresponding switches are closed) . This means that <t>5 
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is describing the situation represented in Fig. 5b while 
<$6 is representing the situation represented in Fig. 5c. 

According to another aspect, the present 
invention is able to discriminate between isolated pixel 
faults and features in the real image. In the case of an 
image of a star covered sky, it should be noted that the 
fact that the image pr<?jected through a lens is never 
perfectly sharp. Even with good lenses, a star image is not 
projected on a single pixel. Always the point like source 
of the start will be smeared out over a central pixel and a 
few neighbours. In a 1-dimensional cross section, a star 
image will look like the image in Fig. 8a, while an 
isolated pixel fault will look like in Fig. 8b. 

In the above simple example, the peak in 
Fig. 8b should be removed, while the peak in 8a should 
remain untouched. 

The advantage is clear, only device faults 
are corrected while normal images are left untouched. The 
operation causes no visible image degradation in faultless 
parts of the image. 

According to this second aspect of the 
present invention, a method to remove an isolated whiter or 
darker pixel from the image is suggested. This method 
consists in limiting the value of every individual pixel 
between an upper and/or a lower bound that is/are derived 
from the values of pixels in the intermediate neighbourhood 
of the said pixel . 

Preferably, the upper and/or lower bounds are 
found by extrapolation of the neighbourhood pixel values 
towards the position of the said individual pixel. The 
upper and/or lower bound are/is a combination of one or 
several such 1-D or 2-D extrapolations done with different 
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methods, and/or from different sides of the said individual 
pixel • 

Preferably, extrapolation is the linear 
extrapolation of a neighbour (Nl) of the said individual 
5 pixel (IP) and the neighbour thereafter (N2) . The 
extrapolated value is calculated as 2*N1-N2 or more 
general: Nl + n* (N1-N2) where the parameter n is a real 
number, typically between 0 and 3. 

According to another preferred embodiment, 
10 the calculation of the upper bounds is performed by 
extrapolating values from the two sides of said individual 
pixels. The advantage is that only the pixels data in 1 
line of an image are required, which saves memory and 
operations and allows straightforward implementation as a 
15 pipelined filter. Also such a filter is able to correct a 
vertical line defect. 

In the example of Figs. 8a and 8b, five 
pixels in a neighbourhood (a 5-pixel "kernel") are taken. 
The experience is that smaller kernels do not yield good 
20 results. Larger kernels may give some improvements compared 
to the 5 -pixel kernel. 
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CLAIMS 

1. An image sensor comprising an array of 
rows (i) and columns (j) of pixels (X± j ) , all the pixels of 
one column of the array being connected to at least one 

5 common pixel output line (lj) having at least one memory 
element (Mj ) and at least a first amplifying element (Aj ) , 
all these amplifying elements (Aj ) being connected to a 

common output amplifier (D) , characterised in that the 
sensor further comprises: 
10 - a second amplifying element (Bj ) on the output of the 
memory element (Mj ) , 

- said common output amplifier (D) having at least two 
input terminals, 

- means (SI) for switching the pixel's signal on the common 
15 output line (lj) and the memory element's signal (Mj ) to 

respectively third and second amplifying elements (Aj and 
B j ) of one column, and 

- means (S2) for switching the two output signals of the 
amplifying elements (Aj , B j ) of one column to 

20 respectively first and second input terminals of said 
common output amplifier (D) ♦ 

2. An image sensor as recited in claim 1, 
wherein said switching means comprise at least one cross- 
bar switch. 

25 3. An image sensor comprising an array of 

columns and rows of pixels (Xij), all the pixels of one 
column of the array being connected to at least one common 
pixel output line (lj) having at least one memory element 
(Mj ) and at least one amplifying element (Aj ) , all these 

30 amplifying elements (Aj ) being connected to a common output 
amplifier (D) , characterised in that before the amplifying 
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element (Aj ) , the common pixel output line (1 j ) is divided 
through switches (S4j and S5 j ) in two parallel circuits, at 
least one circuit having said memory element (Mj ) . 

4. An image sensor as recited in claim 3, 
wherein both circuits have a -memory .element (Msj and Mr j ) 

5. An image sensor comprising an array of 
columns and rows of pixel's (X ± j), all the pixels of one 
column of the array being connected to at least one common 
pixel output line (lj) having at least one memory element 
(Mj) and at least one amplifying element (Aj ) , all these 
amplifying elements (Aj ) being connected to a common output 
amplifier (D) , characterised in that said common pixel 
output line (lj) is being connected through switches (S7j , 
SB j , S9j and S10 j , Sll j ) to a memory element (Cj) and an 
amplifying element <A*j), where the offset of the amplifying 
element is stored on the memory element during a first 
phase of the read-out, and this offset is subtracted from 
the signal of the amplifying element during the second 
phase of the read-out. 

6 . A method of reading out an array of rows 
and columns of pixels (Xj_ j ) of an image sensor according to 
claim 1, comprising the steps of : 

- amplifying the output signals of essentially each pixel 
of one column in the first amplifying element (Aj ) 

thereby obtaining a set of amplified pixel output 
signals, 

- amplifying the reference signals of essentially each 
pixel of one column in the second amplifying element 
(Bj), thereby obtaining a set of amplified pixel 
reference signals , 

- consecutively, for essentially each pixel (i = l, 2, 3) 
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of said column imposing the amplified pixel output 
signal to a first or a second terminal of said common 
output amplifier (D) and imposing the amplified pixel 
reference signal to a second or a first terminal of said 
common output amplifier (D) , while switching the 
amplified pixel output signal to respectively said first 
and said second terminals for essentially each 
consecutive pixel of said column, said reference signal 
being imposed to the other terminal of said common 
output amplifier. 

7 . A method of reading out an array of rows 
and columns of pixels (X^ j ) of an image sensor as recited 
in claim 3, comprising the steps of : 

- sampling the signal in a first phase and storing it in a 
memory element (M-j^ j ) , 

- sampling the signal in a second phase and possibly 
storing it in another memory element, 

- subtracting said first signal from said second signal in 
a unique output circuit (D) . 

8. Method as recited in claim 7, wherein said 
first phase is the reset phase and said second phase is 
after the integration period. 

9 • A method of reading out an array of rows 
and columns of pixels <X i; j) of an image sensor as recited' 
in claim 5, comprising the steps of : 

- during a first phase, calibrating the output of the 
amplifying element to a predetermined value, 

- storing said value in a memory element during the 
application of a first signal of said pixel on the line, 

- during a second phase, applying a second signal of said 
pixel on the line in order to have on the output a signal 
proportional to the difference between first and second 
signals . 
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